Papers with tuning method
PACIT: Unlocking the Power of Examples for Better In-Context Instruction Tuning (2024.findings-acl)
Copied to clipboard
| Challenge: | In-context instruction tuning is a method that fine tunes with supervised instruction data to enhance the instruction following ability of large language models. |
| Approach: | They propose an in-context instruction tuning method that incorporates positive and negative examples into the prompt for better performance. |
| Outcome: | The proposed method outperforms baseline instruction tuning methods on in-domain and out-domain tasks up to 9.16 and 3.14 average ROUGE-L scores. |
Towards an On-device Agent for Text Rewriting (2024.findings-naacl)
Copied to clipboard
Yun Zhu, Yinxiao Liu, Felix Stahlberg, Shankar Kumar, Yu-Hui Chen, Liangchen Luo, Lei Shu, Renjie Liu, Jindong Chen, Lei Meng
| Challenge: | Large Language Models (LLMs) have demonstrated impressive capabilities for text rewriting, however creating a smaller yet potent language model presents two formidable challenges: costly data collection and absence of emergent capabilities. |
| Approach: | They propose a new instruction tuning method to develop a mo-bile text rewriting model that leverages LLM-generated data and heuristic reinforcement learning, eliminating the need for human data collection. |
| Outcome: | The proposed model surpasses the current state-of-the-art LLMs in text rewriting while maintaining a significantly reduced model size using public benchmark EditEval and our new benchmark. |